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Intended for use by managers and developers of Army 
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major component of smart technology — and conventional 
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the current status of intelligent CAI. Abstract concepts from 
cognitive and computer science are discussed in the context of 
specific projects that represent both the long-term promise and the 
current status of smart technology: (1) the exploratory development 
of a prototype smart tutor to train high-level diagnostic and 
troubleshooting skills for repair of a complex reprographics system* 
(2) development of a smart tutor to train technicians in maintenance 
of an Army radar system; and (3) the development and implementation 
of CAI courseware for the U.S. Army Engineer School that takes 
advantage of both current knowledge of human learning and more 
advanced computer technology than is employed in conventional CAI. 
Also discussed are efforts to replace some obstacles to learning that 
are embedded in conventional technical instruction with embedded 
facilitators, which include the creation of problem-solving contexts 
It is anticipated that this application will produce a stream of 
smart facilitators that can be embedded in PLATO or any conventional 
CAI. An outline of projected efforts by the Army Research Institute 
(ARI) to make smart technology a standard tool of the Army training 
community concludes this report. Thirteen references are listed. 
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FOREWORD 



One of ARI s roles as an Army research and development agency is to iden- 
tify important new technologies and encourage their application to Army prob- 
lems. Smart Technology fits into this category. 

Smart Technology represents the application of cognitive and computer 
science to Army training. A major component of this technology is the creation 
ot intelligent tutors (or intelligent computer-assisted instruction (CAl)). 
Intelligent tutors have long been heralded as the next wave of computer-assisted 
instruction. The authors discuss differences between intelligent and conven- 
tional CAI and emphasize the current status of intelligent tutors. Further, 
the authors conclude that intelligent tutoring systems are no longer just the 

toys" of academia, but are beginning to play a vital role in technical train- 
ing. Examples of systems being developed by ARI are provided. 

Also discussed is ARI's effort to produce Smart Technology for CAI devel- 
opers. The authors contend that most conventional CAI is "undersophisticated" 
and that this problem can be corrected by producing Smart Technology aids for 
CAI developers. By way of illustration, a project at the U.S. Army Engineer 
School is discussed. This project is an attempt to apply cognitive science 
findings on problem solving to PLATO CAI for technical courses for engineers. 

Smart Technology may be the best way to meet the training challenge of 
the late 1980s and 1990s without a great and unrealistic increase in money and 
personnel. We at ARI expect that Smart Technology applications will be welcomed 
by training developers as these applications become available. 
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SMART TECHNOLOGY FOR TRAINING: PROMISE AND CURRENT STATUS 



EXECUTIVE SUMMARY 



Requirement : 



u . n+ The AW* training system is straining under greatly increased demands. 
Maintenance and repair of complex, high-technology weapons systems require an 
increasing flow of highly skilled and very specialized technicians. Mobiliza- 
tion requires better training for Reserve Components (training that must be 
delivered at the jhome station one weekend a month and be available in 2-week 
increments during summer duty). Force Readiness requires qualitatively better 
schoolhouse training in less time for the regular Army. Present methods of 
training development and delivery are unlikely to meet these demands without 
an unrealistic increase in resources. 



Product: 



m This paper provides an introduction to a new technology 1 . "Smart Technol- 
ogy is defined as the application of cognitive and computer science to Army 
training problems. Abstract concepts from cognitive and computer science are 
discussed in the context of specific projectsjthat represent both the long- 
term promise and the current status of Smart ^Technology. (Note that the proj- 
ects discussed emphasize two types of Smart Technology: Intelligent Tutors 
(intelligent computer-assisted instruction (CAl)), and cognitive science ap- 
proaches to teaching problem solving in technical domains.) 



Use: 



The report is intended for managers and developers of Army training. In 
recent years cognitive science has grown into a large and fruitful field and 
has reached the point where theories are ripe for training applications. Un- 
fortunately, most of the training development community comes from an intellec- 
tual tradition and moves in professional circles that do not include cognitive 
science. It is our intent to bring cognitive science and the applications of 
Smart Technology to the attention of the training community. We expect that 
such approaches will shed new light on old training problems and that putting 
the ideas of cognitive science to the test of training applications will help 
define the nature and limits of those ideas. 
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SMART TECHNOLOGY FOR TRAINING : PROMISE AND CURRENT STATUS 



OVERVIEW 

The Array's training system is straining under greatly increased demands. 

• Maintenance and repair of complex, high-technolopy weapons systems 
require an increasing flow of highly skilled and very specialized 
technicians. 

• Mobilization requires better training for Reserve Components (train- 
ing that must be delivered at the home station one weekend a month 
and be available in 2-week increments during summer duty) . 

• Force Readiness requires qualitatively better schoolhouse training 
in less time for the regular Army. 

To meet these demands, present methods of training development and delivery 
would require an unrealistic increase in resources. However, an alternative 
exists: Smart Technology. 

In this paper we attempt to explain the jargon and to provide an intro- 
duction to this new technology. Abstract concepts are discussed in the context 
of specific projects that represent both the long-term promise and the current 
status of Smart Technology. ni 



Definitions 



Smart Technology represents the application of cognitive and computer 
science to Army training problems. Cognitive science takes an information 
processing approach to the study of human cognitive processes. Of particular 
importance to trainers is the recent focus on the nature of expertise includ- 
ing expert versus novice, "naive" theories, mental models, and problem solv- 
ing in technical domains. (For a recent review of the importance of cognitive 
science to the national interest, see the National Academy of Sciences, 1983.) 

Smart Technology uses three aspects of the new computer technology. The 
first aspect is the well-publicized availability of sophisticated technology 
at a low cost. The availability of special symbol-manipulating, or LISP, ma- 
chines is the second aspect; these machines were developed specifically as 
tools for research in cognitive science and applications of artificial intel- 
ligence (AI). The third aspect is the AI technology of expert systems and 
intelligent tutors (or computer-assisted instruction CAl). 

In many cases, the cognitive science side of Smart Technology can be ap- 
plied via chalk and talk," paper-based, or standard CAI techniques. However, 
in other cases, the application of Smart Technology requires a delivery vehicle 
that can interactively model a soldier's current knowledge and problem-solvins 
!^ a i e f^ 8 ' f mpare * his ^ th wha * an expert would do, and in real time, design 
and deliver instruction. The new computer technology provides a vehicle capa- 
ble of meeting these needs. (For a "popular" discussion of this new computer 
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technology, see Feigenbaum & McCorduck's book Japan and the Fifth-Generation 
Challenge , 1983.) 



For the past 2 years the Army Research Instituted (ARl) Smart Technology 
for Training team has monitored research in cognitive and computer science to 
determine its applicability to Army training. First, the team has identified 
areas that if nurtured will help meet the training needs of the Army in the 
1990s. At present ARI is supporting long-term development efforts by some of 
the nation's best cognitive and computer scientists. 

Second, there is a critical gap between the skills soldiers must have to 
maintain and repair high-tech systems and the capability of the Army school 
system to train these skills. (Currently, some 80% of these skills are taught 
on the job, not in the school.) Clearly, bold solutions are required to bridge 
this gap. Part of the solution lies in the development of "intelligent" or 
"smart" maintenance tutors capable of delivering high-quality technical train- 
ing in less time and at less cost than current training, ARI is supporting an 
exploratory development effort to build a prototype and is working on a smart 
maintenance tutor for the HAWK Air Defense System. 

Third, there are many findings from cognitive science research that can 
be applied now to improve training developments. Research to apply this knowl- 
edge to improve conventional CAI is currently underway. 



Long-term development projects have been funded through Joint ARI/ONR 
(Office of Naval Research) contracts. Several of these projects relate to the 
development and use of mental models of physical phenomena and devices. 

People use mental models as a basis for predicting outcomes and for plan- 
ning and reasoning. An example of a simple but erroneous mental model is the 
"stove model" of thermostat control. Anyone who has walked into a cold house 
at the end of the day and set the thermostat at 90° is using the stove model. 
This model treats the central heating system as analogous to a gas stove: the 
higher the setting, the higher the flame; the higher the flame, the greater 
the heat; the greater the heat, the faster the pot boils ; hence, the higher 
the thermostat setting, the faster the house heats. In the correct model the 
thermostat is viewed merely as an on/off switch. Setting the thermostat at 
90° will not cause the house temperature to reach 68° any faster than setting 
it at 68°. (See Gentner & Stevens, 1983, for in-depth discussions of mental 
models.) 

One of the projects is examining how mental models of electric circuits 
influence the learning of troubleshooting strategies and, conversely, how 
training in troubleshooting influences the mental models that are developed. 
For example, people often view the flow of electric current as analogous to 
either "flowing water" or "teeming crowds" (Gentner & Gentner, 1983)« These 
models produce different patterns of errors in reasoning about electric cir- 
cuits. Does the effective use of different troubleshooting strategies require 
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the support of different models of current flow? Can one model of current flow 
be found that supports the application of different troubleshooting strategies 
to different problems? The goal is to find some combination of models and 
strategies that results in flexible and effective troubleshooting behavior and 
that can also be easily learned. 

Forming mental models of physical phenomena is so pervasive that people 
will form models based upon incomplete and erroneous information. A wrong 
model may be just incomplete, that is, a preliminary stage in the development 
of more accurate and complete understanding. In contrast, however, misconceived 
models must be unlearned before the correct model can be acquired. For example 
when given the coiled tube shown in Figure 1 and asked to predict the path of a' 

'S * !5 t6 I 6XitS the tub6 ' aboUt half of the college students tested said 
that the ball would continue to curve. A few students said that the ball would 
circle the coil (McClosky, 1 9 8U). Apparently, these individuals have a pre- 
Newtonian impetus" model of this phenomenon. (Passing through the coil im- 
parts a "circular impetus" to the ball, which is gradually lost after the ball 
exits.; McClosky found that even students who had taken a college physics 
course still held some form of impetus theory. Rather than supplanting "naive" 
models of physics, college instruction coexisted or was incorporated into the 
students' preexisting mental models. 

There is a growing appreciation that much of what a good human tutor does 
is to discover students' misconceptions about a domain and provide examples or 
instructions that discredit these intuitive models and lead to the adoption of 
accurate ones. ARI is supporting two projects that examine these aspects of 
mental models and tutoring. The first project supports the development of an 
Al-based medical consultation and explanation system. By interacting with the 
student, the system constructs a dynamic model of the student's knowledge and 
diagnostic reasoning abilities. The system is able to tailor instruction to 
support the student's development of accurate mental models and to help dis- 
credit misconceived models. The second project supports an Al-based system 
that identifies a student-programmer's errors and relates these to the set of 
plans and subplans the student used to write the program. The goal is to use 
errors to diagnose the student's underlying misconceptions about programing 
and then target instruction to correct these misconceptions. 

Another ARI-supported effort is investigating how experts in a domain go 
from a problem statement to making inferences about the problem. The goal is 
to implement these patterns of inference as a computer model and to compare 
the inferences experts make to those that students make. 

The last ARI/ONR project described here is an attempt to develop AI text- 
generation techniques that can be used to teach reading skills to adults. In 
the current prototype, text generation is embedded in a gaming situation that 
requires the student to advance in a fictional organization (for example, an 
Armor Company). The system will generate appropriate written responses to the 
student s moves in the game, administer diagnostic tests for isolating problems 
in reading comprehension, and adapt ively introduce novel materials based on the 
results of the diagnosis. Once the text-generation techniques are developed, 
the content of the gaming situation can be easily altered. The long-term ap- 
plication is to build a family of intelligent tutors in which soldiers can ac- 
quire job knowledge as they improve their reading skills. 
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Figure 1. "Naive physics In accord with medieval impetus theory, half of 
the college students tested predicted that a ball shot through a 
coiled tube would continue to curve after exiting. 



EXPLORATORY DEVELOPMENT 

ARI is supporting Xerox Palo Alto Research Center (PARC) in the explora- 
tory development of a prototype smart tutor to train high-level diagnostic and 
troubleshooting skills for repair of a complex reprographics system. The 
chosen system represents a stable, high-technology testbed (with electronic, 
mechanical, chemical, and electro-optic components). These complex, interact- 
ing technologies require the sophisticated diagnostic and troubleshooting skills 
vital for Army systems in the late 1980s and 1990s. Also important is the fact 
that the system is not classified and has an existing training program with 
which to compare the smart tutor. Measures of on-the-job performance of smart- 
tutored versus conventionally trained technicians will be obtained. The ulti- 
mate proof of the tutor will be whether Xerox uses it for ia-house training. 

The main goal of this effort is not the prototype tutor per se, but the 
software tools, programing environments, and modeling techniques that such a 
tutor requires. Once these tools, environment, and techniques are developed 
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and tested, they will be used to develop a family of smart tutors for training 
maintenance and troubleshooting for Army weapons systems. 

DT Amn Th °r res - ear chers who have watched the development of CAI systems such as 
PLATO and TICCIT will understand the importance of building specialized soft- 
ware tools and programing environments to expand the access to new technology. 
However, the role of mental models and the importance of developing techniques 
to construct such models may not be obvious. 

As an example, a smart tutor for teaching radar maintenance requires four 
different models: first, a straightforward computer model of the radar system 
itself— such device models are becoming standard in many conventional computer- 
based training systems; second, a model of how experts think about the device 
while troubleshooting— an experts' mental model of the device; third, a theory- 
based model of an expert instructor— this model includes pedagogical strategies 
such as when to let a student pursue a wrong answer and when to provide feed- 
back; fourth, the capability to interactively create a model of the student's 
knowledge— including their understanding and misconceptions. (For a discussion 
of smart tutors, see Sleeman & Brown, 1982, and Bregar, 1983.) 

In a smart tutor, these models interact so that at all times the feedback 
provided and problems presented are appropriate to the student's current state 
of knowledge. This technology promises smart tutors that will generate feed- 
back and problems to meet each student's unique needs. This generation is in 
contrast to the best conventional CAI in which students' responses result in 
their being branched down one of a limited number of predetermined paths. 

The HAWK MACH-III 

* The Maintenance Computer for HAWK— Intelligent Institutional Instructor 
J MACH-III) represents the first attempt to build a smart tutor for Army train- 
ing. The HAWK provides over 50% of the air defense for Army units. Current* 
training costs run from $10,000 to $50,000 per radar maintenance trainee, yet 
a recent study identified training as a major problem area that has plagued 
HAWK from the outset. Even a small increase in training effectiveness should 
more than pay back the cost of developing and fielding the MACH-III. 

Our approach at ARI is to pinpoint areas where the MACH-III has the most 
to contribute. We will determine what the major radar maintenance problems 
are and how these problems are addressed by the current training system. Gen- 
erally, we will be interested in how Smart Technology can be used to improve 
existing training methods, and specifically, how a smart tutor can be used 
with greatest leverage. 

The application of smart tutors to HAWK training is ideal. Training prob- 
lems for the HAWK have been recognized as large and costly. Good conventional 
training strategies (that is, chalk and talk, CAI, and videodisk) have been 
tried, but still a large deficit remains. The proponents for the HAWK recog- 
nize this problem and are willing to try a new approach. 

The MACH-III represents a target of opportunity. The high cost of devel- 
opment is more than justified by the potential to improve maintenance training. 
The experience gained in developing the MACH-III will be an important test of 
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newly developed software tools, programing environment, and modeling techniques. 
Experience gained in using the MACH-III will provide guidelines for the most 
effective use of smart tutors. Finally, the experience gained in building the 
MACH-III should cut down the time and costs involved in applying smart tutors 
to other training problems and help transfer smart tutor technology to the Army's 
training development community. 



APPLICATIONS RESEARCH: SMART TECHNOLOGY FOR CAI DEVELOPERS 

It is our contention that most conventional CAI is undersophisticated , 
that is, (a) does not take advantage of what we know about human learning, and 
(b) does not come close to exploiting the power of the computer. We believe 
that the best way for the Army to increase the return on its investment in CAI 
is to put Smart Technology in the hands of CAI developers. 

Toward ohis end, we are working with the U.S. Army Engineer School (USAES) 
to develop and implement CAI in the Engineer Officer Advanced Course (EOAC). 
First, as advisors, we have helped organize and train a CAI courseware cell and 
are now assisting in the formative evaluation of courseware. Second, as re- 
searchers, we have taken a hard look at the nature of the technical courses 
being taught and" the traditional ways (both chalk and talk and CAl) of teach- 
ing them. The Engineer School teaches its Captains a variety of technical 
courses in Civil Engineering (for example, bridge design, soil analysis, flexi- 
ble pavements, and so on). The goal of these courses is to teach Engineer 
Captains how to solve problems in each of these subdomains. 

The simple observation that Engineer Captains are taught how to solve 
problems in various technical domains led us to compare the nature of EOAC 
instruction with findings of cognitive science on problem solving in other 
technical domains (such as physics, chemistry, geometry, and programing). As 
a result of these comparisons, we have begun research designed to apply cogni- 
tive science theories to the design of instruction delivered on a conventional 
PLATO CAI system. 



Obstacles to Learning 

One research goal is to eliminate obstacles to learning embedded in con- 
ventional technical instruction. These obstacles are not unique to the Engi- 
neer School, but are found in all such instruction. 

Identifying the Goal Structure . The goal structure of a problem is the 
"path" taken from the problem statement to its solution. Instruction should 
teach students how to traverse this path but usually does not. In fact, most 
instruction on goal structures contains obstacles to learning. For example, 
Figure 2 presents a traditional two-column proof for a geometry problem. After 
learning a few postulates and theorems, the student is shown one or two of 
these proofs and then is asked to solve problems (Anderson, Boyle, Farrell, & 
Reiser, 198U). 

This "linear" proof structure is very misleading. First, by definition, 
the proof provides no sense of hierarchical relationship among the steps. 
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GIVEN: XY = XZ, 

ZWMY S ZTMZ, 

M midpoint of YZ 

PROVE: YT = ZW 



STATEMENT 

M is midpoint of YZ 
YM = MZ 
XY S XZ 
ZXYZ S ZXZY 

ZWMY S /TMZ 
AWMY S ATMZ 
WY S fZ 
AWYZ S ATZY 
YT = ZW 



REASON 
Given 

Definition of midpoint 
Given 

base angies of 
isoceles triangle 

Given 

Angle-side-angle (ASA) 
Corresponding parts 
Side-Angle-side (SAS) 
Corresponding parts 



Figure 2. Linear (traditional) two-column proof for geometry problems. 

(Adapted from Anderson, Boyle, Farrell, & Reiser, 198I+.) 
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Second, it is not clear to the student whether the order of the steps is ac- 
cidental or essential. These failings are easily overcome by teaching a hier- 
archical proof structure as shown in Figure 3« 

Searching a Problem Space , The problem space includes an individual's rep- 
resentation of the objects in the problem situation, the goal of the problem, 
and the actions that can be performed and strategies that can be used in working 
on the problem. It also includes knowledge of constraints in the problem situa- 
tion: restrictions on what can be done, as well as limits on the ways in which 
objects or features of objects can be combined (Greeno & Simon, 198U, p. h) • 
The search for a problem solution involves a search through some subset of this 
space. For example, Figure h shows the search through a subset of the geometry 
problem space that an expert made while solving the problem given in Figure 2. 
The numbers indicate the sequence of the search. 

The linear proof given in Figure 2 provides no inkling of the false starts 
that the expert went through. The novice is led to believe that all such prob- 
lems are solved in a strictly linear fashion — starting with the givens , and 
generating the next step, until with clock-like precision an answer is found. 
Experts do not solve problems by this process; unfortunately, however, experts 
do teach this way. 

The consequence of these obstacles is that the problem-solving process is 
never explicitly taught but is left to the student to discover alone. This 
"discovery" learning often leads to frustration or failure and can actively 
retard the development of expertise. 



. Facilitating Learning 

We plan to replace embedded obstacles with embedded facilitators. First, 
for selected engineering subdomains, we plan to discover the goal structure 
underlying successful problem-solving and to communicate that goal structure 
to the student. 

Second, typical instruction separates information about the task (usually 
presented by lecture and text) from performance of the task. At best, perform- 
ing the task is regarded as a chance to practice (not acquire) knowledge and 
skills. We plan to revise this procedure and to create problem-solving con- 
texts in which students acquire knowledge and skills as they perform the task. 

Creating problem-solving contexts should facilitate learning for several 
reasons (Anderson, et al., I98U). First, many studies have shown that human 
memory is partially context dependent* If performing a task requires a problem- 
solving activity, then instruction in the facts and theories of the task domain 
should take place in a problem-solving context. Additionally, many concepts are 
hard to understand when presented in the abstract. Teaching these concepts in a 
problem-solving context provides a concrete example of the concept and how it is 
used. Finally, there is the problem of the applicability of information. When 
facts are taught in the abstract, students may be perfectly capable of demonstrat- 
ing that they know these facts (by tests of recall or recognition) but not recog- 
nize that the fact is relevant in a particular context. Students can access mem- 
ory, but they do not know when to apply it. Teaching facts in a problem-solving 
context assists students in learning the "goal relevance" of knowledge. 
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Figure 3. Hierarchical proof structure for geometry problems. (Adapted 
from Anderson, Boyle . Farrell, & Reiser, 198U.) 
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The third way we intend to facilitate learning is by reducing working mem- 
ory failures. Anderson, et al. (198U) argue that working memory failures are 
(a) the major source of errors during learning, (b) an additional limitation on 
the learning rate, (c) a barrier to certain effective types of problem solving 
(such as backwards reasoning in geometry), and (d) a cause of incorrect retrieval 
from long-term memory. Since it is generally accepted that working memory ex- 
pands with expertise, the problem is how to minimize the working memory load of 
novices while they acquire the long-term memory structures prerequisite to a 
larger working memory. (Working memory is viewed as domain dependent. An expert 
with a large working memory in a domain of expertise would not have a large work- 
ing memory available in an unfamiliar domain.) 

Rather than yielding one large product, our applications should produce a 
stream of smart facilitators that can be embedded in PLATO (or any conventional) 
CAI. Many ideas will be rapidly developed and tested using LISP machines. Then 
the most successful ideas will be translated into TUTOR (the language of PLATO) 
as prototype lessons and compared with existing instruction. The prototype les- 
sons that prove effective will be immediately made available to the Engineer 
School. More important, those ideas that prove successful will be immediately 
transferred tc the Engineer School's courseware developers to incorporate into 
their own lessons. In this way we will watch closely the process of technology 
transfer to assist the movement of Smart Technology aids from the laboratory to 
the user. Our ultimate goal is to make all validated Smart Technology aids ac- 
cessible to the Army training community. 



SUMMARY: CONTINUING EFFORTS 

One of ARI's roles as an Army research and development agency is to iden- 
tify important new technologies and encourage their application to Army prob- 
lems. We plan a concerted effort to make Smart Technology a standard tool of 
the Army training community. 

Spreading the word is the keystone of our effort. In recent years cogni- 
tive science has grown into a large and fruitful field and has reached the 
point where theories are ripe for training applications. Unfortunately, most 
of the training development community comes from an intellectual tradition and 
moves in professional circles that do not include cognitive science. Our in- 
tent is to bring cognitive science and its Smart Technology applications to the 
attention of the training community. We expect that Smart Technology will shed 
new light on old training problems and that putting cognitive science ideas to 
the test of training applications will help define the nature and limits of 
those ideas. (For a discussion of the application of artificial intelligence 
to training, see Psotka, 1983. For a discussion of how cognitive science is 
changing the nature of learning theory, see Gray & Hamza, 1983.) 

There are an increasing number of successful training programs based on a 
Smart Technology approach. These programs include such long-standing conundrums 
as literacy training (Wisher, 1983; Wisher & O'Hara, 1981) and technical writing 
(Redish, Felker, & Rose, 1981) . While these programs demonstrate the effective- 
ness of Smart Technology, their developers tend to come from and publish outside 
the traditional training development community. 
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Demonstrating the effectiveness of Smart Technology is also part of our 
continuing effort. The smart tutor for a complex reprographics system is in- 
tended as a demonstration. Arrangements have been made with Xerox to compare 
the effectiveness of the smart "tutor with conventional instruction. Students 
will be Xerox field technicians. Measures of effectiveness will be based upon 
on-the-job performance. 

Also , we expect that our work with the Engineer School and on HAWK MACH III 
will result in impressive demonstrations of Smart Technology's effectiveness. 

Handbooks and guidelines for the application of Smart Technology to train- 
ing do not exist. This lack severely limits the widespread application of Smart 
Technology to training. At present, our efforts at creating guidelines are fo- 
cused on evaluating software tools for building smart tutors. As these tools 
are tested, guidelines will be developed and made available to the training 
community. 

In summary, we see the current examples of Smart Technology as just the 
beginning. Other teams at ARI and at Air Force and Navy laboratories are moving 
quickly to exploit this emerging technology. Smart Technology is the only way 
to meet the training challenge of the late 1980s and 1990s without a great and 
and unrealistic increase in money and personnel. 
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